---
title: Llama CPP
---

<Tip>
**Compatibility**

Only available on Node.js.
</Tip>

This module is based on the [node-llama-cpp](https://github.com/withcatai/node-llama-cpp) Node.js bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp), allowing you to work with a locally running LLM. This allows you to work with a much smaller quantized model capable of running on a laptop environment, ideal for testing and scratch padding ideas without running up a bill!

## Setup

You'll need to install major version `3` of the [node-llama-cpp](https://github.com/withcatai/node-llama-cpp) module to communicate with your local model.

```bash npm
npm install -S node-llama-cpp@3
```
import IntegrationInstallTooltip from '/snippets/javascript-integrations/integration-install-tooltip.mdx';

<IntegrationInstallTooltip/>

```bash npm
npm install @langchain/community @langchain/core
```

You will also need a local Llama 3 model (or a model supported by [node-llama-cpp](https://github.com/withcatai/node-llama-cpp)). You will need to pass the path to this model to the LlamaCpp module as a part of the parameters (see example).

Out-of-the-box `node-llama-cpp` is tuned for running on a MacOS platform with support for the Metal GPU of Apple M-series of processors. If you need to turn this off or need support for the CUDA architecture then refer to the documentation at [node-llama-cpp](https://withcatai.github.io/node-llama-cpp/).

For advice on getting and preparing `llama3` see the documentation for the LLM version of this module.

A note to LangChain.js contributors: if you want to run the tests associated with this module you will need to put the path to your local model in the environment variable `LLAMA_PATH`.

## Usage

### Basic use

We need to provide a path to our local Llama3 model, also the `embeddings` property is always set to `true` in this module.

import LlamaCppBasic from "/snippets/javascript-integrations/examples/embeddings/llama_cpp_basic.mdx";

<LlamaCppBasic />

### Document embedding

import LlamaCppDocs from "/snippets/javascript-integrations/examples/embeddings/llama_cpp_docs.mdx";

<LlamaCppDocs />

## Related

- Embedding model [conceptual guide](/oss/concepts/embedding_models)
- Embedding model [how-to guides](/oss/how-to/#embedding-models)
